WebRTC (Web Real Time Communication) is an open source protocol for establishing media channels between users. Linked hypermedia (graphics, audio, video, plain text and hyperlinks) enrich the content of these channels. Users can accurately cross-link relevant web information, regardless of its location and format, using spatial and temporal descriptions. Examples are: (1) this rectangle area of the real-time video frame is related to this person's home page, (2) this segment of audio conversation is related to this Google map, (3) this segment of the call is related to this Wikipedia page, and (4) this segment of video lecture is related to this part of the YouTube video.
The linked hypermedia creates not only new meaning, but also new communication modalities: users can co-edit a linked Wikipedia page, users can co-browsing a linked Google map, and users can co-view a linked YouTube video.
The linked hypermedia can be created by users or computer programs, such as where each user collaboratively contributes his knowledge, a computer program brings in new information according to scheduled topics, or a computer program augments conversation in real-time (e.g., based on automated face recognition and/or automated speech recognition).
Real-time media access and control in web browsers can be accomplished or performed using various interfaces/specifications, including WebRTC application programming interface (API), HTML5 media API, Stream Processing API, Web Audio API, Mozilla Audio Data API, Media Controller API, Media Capture API, and HTML5 Media Capture API.
Multimodal interactions in real and virtual environments may be accomplished or performed in accordance with various specifications, such as World Wide Web (W3C) VXML 3.0, W3C Multimodal Architecture and Interface, W3C Emma, W3C SCXML, W3C InkML, W3C EmotionML, W3C SMIL, and Web Real-Time 3D.
Various problems exist in the current art in this field. Media URIs generated by a web browser are local and only resolvable by the browser, and are temporary and will become invalid after the browser exits. In addition, two communicating web browsers generate different URIs for related media streams. For example, browser A assigns the video from camera A_URI for a local view, while browser B assigns the video from A's camera B_URI for peer view. In this case, the A_URI and the B_URI have no common part for correlation. Further, in a collaborative multi-user conference, when an action is performed on A_URI at browser A, how is it known that it should be performed on B_URI at browser B, when each browser has many media streams?
The present disclosure provides various methods, mechanisms, and techniques to establish correlations between local media URIs during a WebRTC call, record the correlations as resource description framework (RDF) <mirror> predicates in an RDF store, and translate RDF graphs exchanged between web browsers based on the RDF <mirror> predicates.